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ABSTRACT 

Most academic libraries undertake the collection and 
reporting of statistical information on activity levels for many 
aspects of both technical and public services. Data on reference 
activity are used for a variety of purposes within the library, 
primarily to facilitate planning and improve delivery of reference 
services, and also externally, where they are used for broad 
inter-institutional comparisons which often have a direct impact on 
operations. The Canadian Association of College and University 
Libraries (CACUL) Task Force on Reference Statistics was established 
in October 1985 with a dual mandate: (l) to develop definitions, 
forms, and procedures, and to conduct a pilot study to establish a 
body of data on reference activities in Canadian academic libraries; 
and (2) to develop a means of measuring patron satisfaction with 
reference service. In 1990, the Working Group on Reference Statistics 
was formed to address the first part of the mandate. Questionnaires 
were developed and sent to academic libraries, the results of which 
led to the development of draft definitions and forms. A pilot 
project to test the efficacy and acceptability of definitions and 
forms was undertaken in March 1993. Responses were received from 10 
universities (66.67.) and 15 colleges (93.87.). Results are discussed. 
Appendices include: a pretest questionnaire; a survey questionnaire; 
instructions; definitions; forms; and a questionnaire distributed 
after the test period. (SWC) 
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INTRODUCTION 



V 



The CACUL Task Force on Reference Statistics was established in October 1985 with a dual 
mandate: 



To develop definitions, forms, and procedures, and to conduct a pilot study to 
establish a body of data on reference activities in Canadian academic libraries 



and 



To develop a means of measuring patron satisfaction with reference service. 

The original task force focussed on the second of these two. During its tenure, the Task Force 
investigated instruments of evaluation, and examined, tested, and endorsed the Wisconsin-Ohio 
Methodology developed by Charles Bunge and Marjorie Murfin. The results of the Task Force's 
work were published in three CACUL Occasional Papers. "Evaluation of Reference Services at 
the University of Waterloo" (1985); "Wisconsin-Ohio Reference Evaluation at the University of 
Waterloo" (1989); and "Comparison of Reference Success in Canadian and American Libraries 
Participating in the Wisconsin-Ohio Reference Evaluation Process" (1989). The Task Force 
recommended the Wisconsin-Ohio Reference Evaluation Process as an economical, well- 
designed and effective means of measuring the quality of reference service. 

The Task Force submitted its final report to the CACUL Executive in June 1989. The Executive 
felt that continuing work was required with respect to the first-named initiative - to establish a 
body of data on reference activities in academic libraries and to develop definitions, forms, and 
procedures. 

In January 1990 a new CACUL Working Group on Reference Statistics was struck. Its members 
were Sandra Black, Mohawk College, Andrea Glover and Jim Kelly from the University of 
Lethbridge, and Jane Philipps, Queen's University. In initial discussions, Working Group members 
identified some of the issues to be addressed. Why do we keep reference statistics? Do they 
serve a useful purpose? How are they collected and reported? What are they used for? What 
should and could they be used for? How is reference activity defined? Are standard definitions 
and forms that can be used in all Canadian academic libraries (a) desirable and (b) feasible? 

One of the members of the original task force wrote, that within the term of its mandate, the task 
force had not had time to 

come to grips with the several cans of worms associated with the more mundane 
statistics collected at reference desks on a daily, or at least regular basis. They did 
not have time to pick up the canopener, or even open the cupboard to examine 
the cans. 

In its second incarnation, the Working Group has opened the cupboard, opened the cans, 
examined and made some recommendations on the handling of the worms. 
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METHODOLOGY 






In November 1991 the Working Group developed a survey designed to collect information on 
current activities relating to the collection, reporting, and use of reference statistics in Canadian 
academic libraries (Appendix 1). The questionnaire was pretested by a group of 12 college and 
10 university libraries. On the basis of comments from the pretest participants, the questionnaire 
was revised and a full-fledged survey was undertaken in the spring of 1992 (Appendix 2). 

Questionnaires were sent to 42 university libraries of which 33 (or 78.6 percent) responded and 
to 76 college libraries of which 43 (or 56.6 percent) responded. The number and quality of 
responses provided evidence of the level of interest in the issue of reference statistics. Many 
respondents took the time to express their concerns and desires in great and thought-provoking 
detail. 

The information collected in the survey, particularly the comments regarding strengths and 
weaknesses of current practice and suggestions for possible improvement, led to the development 
of draft definitions and forms. A pilot project to test the efficacy and acceptability of definitions 
and forms was undertaken in March 1993. Instructions, definitions, and forms were mailed to 15 
university and 16 college libraries for testing in the busy March period (Appendices 3 to 5). 
Following the test period participants completed a questionnaire (Appendix 6) and provided 
evaluative comments on the forms and definitions used in the test. Responses were received 
from 10 universities (66.6 percent) and from 15 colleges (93.8 percent). Definitions and forms 
were revised on the basis of pilot project responses. 

The Working Group submitted its Final Report recommending the adoption and promotion of the 
definitions and forms as a national standard to the CACUL Executive in February 1994 (Appendix 
7). The CACUL Executive endorsed the Working Group's recommendations. 

THE SURVEY - RESULTS AND DISCUSSION 



Collection of data on reference activities 

The following tables identify the questions asked in the survey of current activities related to the 
collection, reporting and use of reference statistics in Canadian academic libraries and show the 
percentage of responses received, broken down by college and university libraries. These data 
and the comments of survey respondents provide an overview of the major issues and concerns 
raised by repondents to our survey. The issues raised are remarkably similar across the board 
and the results offer a picture of the situation in 1992. 

Information on responses to the questions relating to collection of data on reference activities are 
presented in Table 1. Five university libraries (12 percent of the original sample) and 16 college 
libraries (21 percent of the original sample) did not, at the time of the survey, collect information 
on reference activities. 

29.4 percent of college libraries and 56.7 percent of university libraries indicated that they had 
a written definition of a reference transaction. In actual fact, the majority of definitions provided 
represented descriptions of the various categories by which reference transactions are classified 
rather than any prescriptive definition of what a reference transaction is. 
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Table 1 . Responses to questions on the collection of reference statistics broken down by 
college and university libraries 



QUESTION 


COLLEGE 


UNIVERSITY 


1. Do you have a written definition of a "reference" 
transaction? 


Yes: 29.4 


Yes: 56.7 


2. A formal record of the number of reference 






transactions is maintained at: 






Reference/I nfonmation Desk 


91.7 


100.0 


Public service points other than a 


69.7 


56.7 


Reference/Information Desk 






Locations other than designated service points 


11.4 


16.7 


3. Recorded transactions are categorized to 






indicate level of complexity in terms of: 






Time required to complete the transaction 


54.3 


20.0 


Definition reflecting complexity of intellectual content 


40.0 


53.3 


Other 


8.3 


10.0 


4. Recorded transactions are categorized by: 






Hour of the day 


32.4 


66.7 


Time of day (e.g. morning, evening) 


57.1 


66.7 


Day of the week 


80.0 


90.0 


Patron category 


17.1 


16.7 


Communication mode 


28.6 


56.7 


Other 


11.4 


30.0 


5. There are categories of questions handled at the 
Reference/information Desk which are not 
recorded. 


39.4 


41.4 


6. A formal record of transactions not completed 
to the satisfaction of either or both patron and the 
library staff member is maintained. 


45.7 


6.7 
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Non-prescriptive descriptions of categories of reference transactions are exemplified below: 

"Quick Reference: Any questions that can be answered without leaving the desk, 
including directions and referrals. Reference: Any question that takes one away 
from the desk and which is anwered in less than fifteen minutes. Extended 
Reference: Any question that takes one away from the desk and which requires 
more than fifteen minutes to answer." 

Those institutions which supplied a "prescriptive" definition generally employed some version of 
the definition which provided the Working Group with its final model. 

In general, "reference" activities, as defined by survey respondents, were broadly categorized as 
directional, short reference, and long reference, with the distinction between "short" and "long" 
based on time required to complete the transaction. A number of institutions used a separate 
category for instructional exchanges, sometimes further divided as quick instruction and long 
instruction. Many included a category for extended reference. A large number of institutions 
included directional exchanges under the rubric of reference, however, 41 .4 percent of university 
libraries and 39.4 percent of college libraries indicated that directional questions answered at the 
reference desk were not recorded as reference activity. 

In the majority of cases, "definitions" reflecting complexity of intellectual content of queries were 
also expressed in terms of time required to complete the transaction. There was a great deal 
of variation in the amount of time used to distinguish between "easy" and "hard" reference 
queries. Time factors range from three minutes to two hours with five and fifteen minutes being 
the most commonly used cutoffs between simple and complex reference transactions. In addition 
to categorization by question type, information on reference transactions was also generally 
broken down by time, communication mode or patron type. 

Although as a rule there is a great deal of similarity between responses from colleges and 
universities, the difference in Question 6 is worthy of note. Only 6.7 percent of university libraries 
maintain a record of transactions not completed satisfactorily compared to 45.7 percent of college 
libraries. Many college respondents indicated that records of dissatisfaction were used to identify 
and remedy gaps in the collection, cataloguing problems, and lack of subject expertise on the part 
of reference staff handling the question. 

Reporting of data on reference activities 

Information on the reporting of data on reference activities is presented in Table 2. Despite the 
level of detail used in recording/collecting data on reference activities, very few colleges or 
universities employed the same level of detail in reporting on the data collected. Data collected 
were primarily reported in terms of total numbers of transactions. Detailed information relating 
to time of day or patron category or communication mode or complexity appeared to be used for 
internal purposes only. 
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Table 2 . Responses to questions on the reporting of reference statistics 



QUESTION 


College 


University 


1. Reference statistics are reported: 






Monthly 


82.9 


75.0 


Annually 


54.3 


64.3 


Other 


14.3 


21.4 


2. Reference statistics are reported to: 






Reference staff 


71.4 


75.9 


Library administration 


94.3 


93.3 


College administration 


74.3 


53.3 


External organizations 


22.9 


30.0 


3. Reference transactions are reported: 






As total number of transactions 


97.1 


100.0 


By hour of the day 


20.6 


14.8 


By time of the day (e.g. morning) 


29.4 


17.9 


By day of the week 


39.4 


25.0 


By patron category 


11.8 


10.3 


By communication mode 


14.7 


42.9 


Other 


32.4 


25.0 



Table 3 . Responses to questions on the use of reference statistics 



QUESTION 


College 


University 


1. Reported statistics are used to: 






Evaluate reference service 


43.7 


37.9 


Determine levels of staffing 


75.0 


86.2 


Determine levels of budget support 


46.9 


26.7 


Determine hours of service 


75.0 


86.7 


Evaluate instructional programmes 


21.9 


26.7 


Identify collection development needs 


53.1 


30.0 


Other 


6.2 


3.3 
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Use of reference data 

Although we commonly refer to the collection of reference statistics, the point has been made 
that what we collect are data which only become statistics when we do something with them. 
Table 3 identifies the use to which data on reference activity were put. The survey concluded 
with three open-ended questions. "Do you feel that the collecting and reporting of statistics on 
reference activity is useful and/or meaningful?" "Are you satisfied with your present procedures?" 
"How might the collection, reporting, and use of statistics relating to reference activity be made 
more useful and/or meaningful?" These questions garnered many thoughtful and thought- 
provoking responses which can be discussed in the context of Table 3. 



Reference statistics were primarily used for planning purposes internal to the library to determine 
hours of service and appropriate staffing levels. With this in mind, many respondents indicated 
that though reference statistics serve as indicators of activity levels and workload, they are 
approximations at best. As one respondent noted "a busy reference desk never keeps accurate 
statistics." In fact, numbers of transactions were reported least accurately when demand for 
service was highest. Staff are simply too busy to record all the questions they handle, particularly 
those they respond to in transit to or from the desk. Another respondent remarked that "we all 
have times when we are so busy that we just throw a bunch of tick-marks into the appropriate box 
at the end of the day." In addition, queries handled at points other than the reference desk are 
rarely included. A number of people also commented that not all staff were equally concerned 
with maintaining an accurate record of activity. In addressing this point, it was noted that 
reference desk staff need to be convinced that the collection of accurate information is important 
and that statistics will never be meaningful unless they are accurate. Staff need to know the 
reasons for collection of data and that statistics can and will be used in a meaningful way. 

26.7 percent of university libraries and 46.9 percent of college libraries indicated that reported 
statistics are used to determine levels of budget support. Many commented that, though in theory 
the figures are used to justify existing levels of budget support or as arguments for increased 
funding for staffing, in actuality, the numbers rarely have any impact. It was noted that non-library 
administrators simply do not understand what the figures represent. One respondent remarked 
that unless one has experienced it first hand, one cannot understand what it means to answer 
25 questions in an hour. In order to be meaningful to decision-makers, the bald numbers must 
be accompanied by a description of what reference activity entails and what it contributes to the 
academic enterprise. If we believe that the collection and reporting of statistics related to 
reference activity are of use, we need to find ways of making statistics relevant to 
administrators who are not knowledgeable about reference and/or about libraries and are 
only concerned with the bottom line. 

Van House, Weil, and McClure suggest a number of ways of analyzing and correlating data to 
produce useful and meaningful results. These include comparing the annual number of reference 
transactions to cost data for the provision of reference service, dividing the number of reference 
transactions by the number of total library uses to determine the ratio of reference questions 
received to library uses, dividing the number of reference transactions by FTE staff available to 
provide service, to measure workload per staff member, and dividing the total number of 
reference transactions by the hours that reference service is available to determine how 
extensively the service is used. 
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A number of respondents indicated that reported statistics are used to evaluate reference service 
and to evaluate instructional programmes. Such evaluation appeared to be qualitative, based on 
anecdotal evidence, rather than quantitative. One respondent commented that in order to use 
statistics to evaluate more than just staffing, budget and hours we need to "identify other variables 
that have not been part of the standard recipe ... one beneficial effect of rethinking our motives 
is that, before launching into a project to collect statistics, we require ourselves to define and 
examine our goals very carefully. We might discover in some cases that we already have the 
information we need to make a decision." 

For the most part, and with some disclaimers, respondents felt that the recording of data on 
reference activities is useful. In terms of making the collection and reporting of reference 
statistics more meaningful, there were two recurring themes: 

standardized definitions, forms, and methodologies are required in order to permit 
meaningful comparisons within and among institutions 

a means of objectively assessing the complexity of reference questions is 
required. 

THE PILOT PROJECT - RESULTS AND DISCUSSION 

The pilot project aimed to test the utility of draft definitions and forms for the collection of data on 
reference activity. 15 university and 16 college libraries participated in the pilot project, working 
with forms and definitions (Appendices 4 and 5) and evaluating their efficacy. Following the test 
period participants completed a questionnaire (Appendix 6) and provided evaluative comments 
on the forms and definitions used in the test. Responses were received from 10 universities 
(66.6 percent) and from 15 colleges (93.8 percent). A summary of responses to the "Pilot 
Project Evaluation Questionnaire" is presented in Table 4. Definitions and forms were revised 
on the basis of pilot project responses. 

Definitions 

The majority of repondents to the 1992 CACUL Survey of Activities Relating to the Collection, 
Reporting, and Use of Reference Statistics in Canadian Libraries identified the need to (1) 
distinguish between reference and non-reference transactions and (2) to provide some measure 
of a reference transaction's intellectual complexity. The Working Group proposed the delineation 
of three transaction categories - basic reference transactions, complex reference transactions, and 
non-reference transactions variously labelled as facilitative or directional. 

Since the American Library Association's definition of a reference transaction is the one currently 
in use in a number of Canadian academic libraries, the Working Group adopted it as the model 
definition for reference transactions proper. To address the need to make a further qualification, 
of reference transactions on the basis of complexity, the Working Group expanded the model 
definition by adding two subcategories - basic and complex. 

The ALA’s parallel definition of a directional transaction was also considered a valid model but 
the term "directional" was deemed too narrow since literal interpretation of the word could result 
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in the inclusion of some types of non-directional yet non-reference queries into the reference 
category. Consequently, it was decided to label all non-reference transactions "facilitative" to 
include the many kinds of time-consuming assistance that are given in academic libraries which 
are still not "reference." The working definitions for the pilot project are provided in Appendix 
4. 

The working definitions employed in the pilot project were modified on the basis of comments and 
suggestions from the pilot project participants. In general, participants reported that the definitions 
permitted clear distinctions between non-reference and reference transactions and between basic 
and complex reference transactions. Although several respondents preferred the label 
"directional," the majority felt that the label "facilitative" was more encompassing. The 
"facilitiative" category provided a means of tracking assistance provided to users at service points 
other than the reference desk and for monitoring the increasing number of transactions involving 
assistance with computers including downloading, troubleshooting, disk formatting and so on. 

A number of respondents reported that, initially, the proposed CACUL definitions required 
reference staff to analyze the type of assistance that had been given, but that by the end of the 
month the process had become quite automatic and that 99 percent of transactions fell easily into 
one category or another. Several respondents felt that an additional category for questions 
requiring considerable research on the part of reference staff member was required. The 
proposed distinction between "basic" and "complex" generated the most comment. Many 
respondents noted that the level of knowledge and experience of individual staff members, the 
library literacy level of the user, and the availability of "the perfect source" have an impact on 
whether a transaction is considered "basic" or "complex." What is "basic" for one staff member, 
may be complex for another. What is easy to explain to one user, may be extremely difficult to 
explain to another and therefore requires more effort and expertise on the part of the individual 
instructor. What is complex one. day, may be basic the next. What would be a basic question, 
given the availability of a particular reference source, becomes complex if that source is not 
available. Finding an address, for example, might require consultation of one source, or may 
require considerable innovation and resourcefulness on the part of the reference staff member. 



On the basis of feedback from pilot project participants, adjustments were made to the 
definitions. The Working Group focussed on the question of the purpose for keeping straight 
statistics and concluded that at the most fundamental level, the distinction between "basic" and 
"complex" is meant to provide a measure of the level of effort expended by the individual staff 
member rather than a measure of a given question's conceptual complexity. The issue of 
question content and the handling of questions by reference staff in the context of evaluation of 
reference service, is a separate one requiring other approaches suggested in the earlier CACUL 
Task Force which reported in June 1989. The revised definitions include a statement placing the 
emphasis, for statistical purposes, on how the question was handled rather than on the question 



The CACUL Executive has endorsed the following definitions and recommended their adoption 
in all college and university libraries. The Final Report of the Working Group (Appendix 7) 
included a further recommendation that the definitions be adopted by the Canadian Library 
Association as a national standard. 



itself. 
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Information Exchanges or Transactions 



FACILITATIVE 

An information contact that facilitates the use of the library but which does not involve the 
knowledge, use, recommendation, interpretation, or instruction in the use of any information 
sources other than those which describe the library, such as schedules, floor plans, handbooks, 
policy statements. Examples of facilitative transactions include giving directions for locating staff, 
users, physical features within the library, etc., explaining library policies/procedures, or giving 
assistance of a nonbibliographic nature, with machines, e.g. printing from CD-ROMS, downloading 
to disk, changing ribbons or paper. 



REFERENCE 

An information contact that involves the knowledge, use, recommendation, interpretation, or 
instruction in the use of one or more information sources. Information sources include printed and 
nonprinted materials, machine-readable databases (including computer-assisted instruction, 
catalogues, and other holdings records), through communication or referral, other libraries and 
institutions, and persons both inside and outside the library. When a staff member utilizes 
information gained from previous use of information sources to answer a question, it should be 
treated as a reference exchange, even if the source is not consulted again during the transaction. 



BASIC: A request for information which can be easily provided, generally with 
reference to one or two standard information/reference sources and including basic 
instruction if required. As a general guideline, takes up to five minutes. The 
emphasis is on the question as it was handled, not on the knowledge of the 
questioner or the experience of the staff member. 

COMPLEX: A request for information which requires consultation of a variety of 
information/reference resources and/or the exercise of high-level query negotiation, 
information-seeking, and instructional skills. As a general guideline, takes more 
than five minutes. The emphasis is on the question as it was handled, not on the 
knowledge of the questioner or the experience of the staff member. 
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Table 4. Responses to the pilot project evaluation 



QUESTION 


YES 


NO 


1. The definitions clearly distinguish between reference and 
non-reference (directional or facilitative transactions). 


95.5 


4.5 


2. The definitions clearly distinguish between basic and 
complex reference transactions. 


81.8 


18.2 


3. The definitions are appropriate and we would use them. 


81.8 


18.2 


4. We would use the following forms 






a. Daily (by time of day) 


40 


60 


b. Daily (by hour) 


40 


60 


c. Weekly 


11 


89 


d. Monthly 


11 


89 


5. We would prefer to collect statistics on a 






a. Daily basis 


80.0 


20.0 


b. Using a valid sampling technique 


20.0 


80.0 



Forms 

The forms met with the general approval of pilot project participants and several respondents 
confirmed that the forms were easily modified to enable collection of data in additional 
categories. One respondent noted that the fact that the statistics sheet was divided into three 
equal parts seemed to promote accuracy. 

The forms employed in the pilot project were designed to measure two variables only - type of 
question and period of collection i.e. daily (by hour), daily (by time of day), weekly and monthly. 
The intent was to develop a set of base forms which identify the major categories of information 
to be collected but which can be easily modified to meet the particular needs of individual 
reference departments. For example, respondents to the survey indicated that data on variables 
such as patron type or communication mode are important service measures. These additional 
variables can be easily accommodated withfn the context of the three question categories. 
An example of a form which breaks out telephone and in-person queries is included in Appendix 
5. Similarly, the forms can be modified to reflect the timing of particular reference desk shifts or 
schedules. 
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Sampling 



A number of respondents indicated that they had used sampling to clarify particular issues. For 
example, one institution noted that they had maintained a count of e-mail and telephone queries 
over three one-week periods in order to establish a basis for comparison at a later time. Others 
expressed an interest in sampling as an alternative to daily tallying of data. Proponents of 
sampling suggest that collecting information by sampling will produce much more statistically 
accurate results. For example, Edinger and Falk (1981:267)) note in their discussion of statistical 
sampling noted that 

It was also believed that if librarians were asked to record statistics for only eight 
weeks per year, they would be particularly aware of the project during those two- 
week periods and would therefore be especially conscientious about recording 
reference-desk queries. 



Van House, Weil, and McClure suggest the following, noting however, that with this approach it 
is difficult to extrapolate the data from the sample period to provide an accurate figure for the 
entire academic year. At best, this data should be reported for the period of time it is collected. 

Select a reasonably representative time period -that is, a busy time but (in 
academic libraries) not during midterms or finals or periods of other unusual 
activity. During the sample period ask library staff to count reference transactions 
at all relevant service points. We recommend that you collect data all the hours 
that reference service is provided, for one or preferably two weeks. 



The Report of the Task Force on Canadian Library Statistics (1988) suggests the following: 

Choose a typical day for each of the twelve months of the year, and on that day 
count (1) the number of people entering the library, and (2) the number of 
questions answered. From these data calculate the average number of questions 
answered per person entering the premises. Then multiply this number by the 
total number of people entering the premises annually. 



Additional methodologies are described by Edinger and Falk (1981) and Halperin (1974 and 
1978). 
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CONCLUSION 



Most academic libraries undertake the collection and reporting of statistical information on activity 
levels for many aspects of both technical and public services. Data on reference activity are 
used for a variety of purposes within the library and within the institution, primarily to facilitate 
planning and improve the delivery of reference services. The data are also reported to external 
organizations and associations where they are used for broad level interinstitutional comparisons 
which often have a direct impact on operations within individual institutions. 

Comparisons within and among institutions are only of value if it is clear that all institutions are 
collecting and reporting the same information. Therefore, one of the Working Group's primary 
mandates was to develop a standard set of definitions that could be used by all colleges and 
universities. At the same time, the definitions had to support internal collection and reporting 
requirements. 

The Working Group believes that the use of the definitions and forms as recommended will result 
in a level of internal and interinstitutional comparability that has hitherto been lacking. They will 
also allow for more specific detail to be collected for local purposes - within the categories - 
without distorting their comparability. 
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APPENDIX 1 Pretest Questionnaire 



CACUL Survey of Activities Relating to the Collection 
Reporting and Use of Reference Statistics 
in Canadian Academic Libraries 



A: INSTITUTIONAL BACKGROUND 

This information will be considered confidential and will be used for analysis purposes only. 

1 . Name of college/university: 

2. Name of library 

3. Name of contact person: 

4. Address: 



5. Telephone: ( ) Fax: ( ) 

E-mail: 

6. Would you be willing to be interviewed by telephone as a follow-up to your questionnaire responses? 

Yes No 

7. Would you be willing to participate in a pilot project to test the usefulness of proposed standard 

definitions, forms, and procedures for recording and reporting reference activities in Canadian academic 
libraries? Yes No 



Signature: 



Date: 



Please Return To: E. Jane Philipps By: NOVEMBER 15, 1991 

Biology Library, Earl Hall 
Queen's University 
Kingston, Ontario 
K7L 3N6 

Telephone: (613) 545-6846 
Fax: (613) 545-6189 
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B: COLLECTION: 



Please attach a sample of all forms and definitions used in the collection of reference statistics. 

1 . Do you maintain a formal record of the number of reference transactions which take place at any of 
the following?: 

a. Reference/Information Desk Yes No 

b. Public service points other than a Reference/Information Desk Yes No 

c. Locations other than designated service points. Please explain. Yes No 

Comments: 



2. Are recorded transactions categorized to indicate the level of complexity in any of the following 
ways?: 

a. Time required to complete the transaction. Please specify. Yes No 

b. Definition reflecting complexity of intellectual content. Yes No 

c. Other. Please specify. Yes No 

Comments: 



3. Are recorded transactions categorized in ways other than those related to level of complexity?: 



a. By hour of the day 

b. By time of day (e.g. morning, evening) 

c. By patron category . Please specify. 

d. Other. Please specify. 

Comments: 



Yes No 

Yes No 

Yes No 



BEST COPY AVAILABLE 
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B: COLLECTION (continued) 



4. Are there categories of transactions handled at the Reference/Information Desk which are not 
recorded? Yes No 

Comments: 



5. Do you maintain a formal record of transactions not completed to the satisfaction of either or both the 
patron and the library staff member? If yes, how is this record used? Yes No 

Comments: 



C: REPORTING 

Please attach a sample of all forms used in the reporting of reference statistics. 
1 . How often are reference statistics reported? 

Monthly Annually Other 

Comments: 



2. To whom are reference statistics reported? 

a. Library administration 

b. College/University administration 

c. External organizations. Please specify. 

Comments: 



Yes No 

Yes No 

Yes No 
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C: REPORTING (continued) 



3. In what format are reference statistics reported? 

a. Total number of transactions Yes No 

b. Broken down by type Yes No 

c. Broken down by time Yes No 

d. Other. Please specify. Yes No 

Comments: 



4. Does the manner in which reference statistics are reported reflect the manner in which they are 
collected? (For example, you maintain a very detailed record but report only cumulations.) 

Yes No 

Comments: 



D: USE : 

1. Are reported reference statistics used: 

a. To evaluate Information/Reference service? 

b. To determine levels of staffing? 

c. To determine level of budget support? 

d. To determine hours of service? 

e. To evaluate instructional programmes? 

. Other. Please specify. 

Comments: 



Yes No 

. Yes No 

. Yes No 

Yes No 

_ Yes No 

Yes No 
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D. USE (continued) 



2. Do you feel that the collecting and reporting of statistics on reference activity is useful/meaningful? 



3. Are you satisfied with your present procedures for collecting and reporting statistics on reference activity? 



4. How might the reporting of reference activity be made more useful/meaningful? 



APPENDIX 2 . Survey Questionnaire 



CACUL Survey of Activities Relating to the Collection 
Reporting and Use of Reference Statistics 
in Canadian Academic Libraries 



A: INSTITUTIONAL BACKGROUND 

This information will be considered confidential and will be used for analysis purposes only. 

1 . Name of college/university: 

2. Name of library 

3. Name of contact person: 

4. Address: 



5. Telephone: ( ) Fax: (__) 

E-mail: 

6. Would you be willing to be interviewed by telephone as a follow-up to your questionnaire responses? 

Yes No 

7. Would you be willing to participate in a pilot project to test the usefulness of proposed standard 

definitions, forms, and procedures for recording and reporting reference activities in Canadian academic 
libraries? Yes No 



Signature: 



Date: 



Please Return To: E. Jane Philipps By: April 15, 1992 

Biology Library, Earl Hall 
Queen's University 
Kingston, Ontario 
K7L 3N6 

Telephone: (613) 545-6846 
Fax: (613) 545-6189 
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B: COLLECTION : 

Please attach a sample of all forms and definitions used in the collection of reference statistics. 

1 . Do you have a written definition of a "reference" transaction? If yes, please include your definition. 

Yes No 

2. Do you maintain a formal record of the number of reference transactions which take place at any of 
the following?: 

a. Reference/Information Desk * Yes No 

b. Public service points other than a Reference/Information Desk Yes No 

c. Locations other than designated service points. Please explain. Yes No 

* Please indicate whether you have established separate "information" and "reference" desks. 
Comments: 



3. Are recorded transactions categorized to indicate the level of complexity in any of the following 
ways?: 

a. Time required to complete the transaction. Please specify. Yes No 

b. Definition reflecting complexity of intellectual content. Yes No 

c. Other. Please specify. Yes No 

Comments: 
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B. COLLECTION (continued) 



4. Are recorded transactions categorized in ways other than those related to level of complexity?: 



a. By hour of the day 

b. By time of day (e.g. morning, evening) 

c. By day of the week 

d. By patron category . Please specify. 

e. By communication mode (e.g.in person vs telephone). Please specify, 
d. Other. Please specify. 

Comments: 



Yes No 

Yes No 

Yes No 

Y es No 

Yes No 



5. Are there categories of transactions handled at the Reference/Information Desk which are not recorded? 
If yes, please explain. Yes No 

Comments: 



6. Do you maintain a formal record of transactions not completed to the satisfaction of either or both the 
patron and the library staff member? If yes, how is this record used? Yes No 

Comments: 
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C: REPORTING 



Please attach a sample of all forms used in the reporting of reference statistics. 

1 . How often are reference statistics reported? 

Monthly Annually Other 

Comments: 



2. To whom are reference statistics reported? 

a. Reference staff 

b. Library administration 

c. College/University administration 

d. External organizations. Please specify. 

Comments: 



Yes No 

Yes No 

Yes No 

Yes No 



3. In what format are reference statistics reported? 

a. Total number of transactions 

b. Broken down by hour of day 

c. Broken down by time of day (e.g. morning, evening) 

d. Broken down by day of the week 

e. Broken down by patron category 

f. Broken down by communication mode (e.g. in person vs telephone) 

g. Other. Please specify. 

Comments: 



Yes No 

Yes No 

Yes No 

Yes No 

Yes No 

Yes No 

Yes No 
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D: USE: 



1 . Are reported reference statistics used: 

a. To evaluate Information/Reference service? Please explain. 

b. To determine levels of staffing? 

c. To determine level of budget support? Please explain. 

d. To determine hours of service? 

e. To evaluate library instructional programmes? Please explain. 

f. To identify collection development needs? 

g. Other. Please specify. 

Comments: 



Yes 


No 




Yes 


No 




Yes 


. No 






Yes 


No 




Yes 


No 




Yes 


No 




Yes 


No 



2. Do you feel that the collecting and reporting of statistics on reference activity is useful/meaningful? 



D. USE (continued) 

3. Are you satisfied with your present procedures for collecting and reporting statistics on reference activity? 



4. How might the reporting of reference activity be made more useful/meaningful? 
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APPENDIX 3 . Pilot Project Instructions 



February 15, 1993 



1 ~ 

2 ~ 

3~ 

4~ 

5~ 



Dear 6~ 

Thank you for agreeing to participate in the CACUL Working Group on Reference Statistics 
pilot project. The purpose of the project is to test the utility of draft definitions and forms for 
the collection of statistics on reference activity which have been developed on the basis of 
responses to the April 1992 CACUL Survey of Activities Relatino to the Collection. Reporting, 
and Use of Reference Activities in Canadian Academic Libraries . 

The majority of respondents identified the need to (1) distinguish between reference and non- 
reference transactions and (2) to provide some measure of a reference transaction's 
intellectual complexity. The Working Group proposes the delineation of three transaction 
categories - basic reference transactions, complex reference transactions, and non-reference 
transactions variously labelled as facilitative or directional. 

The Working Group recommends the adoption of the following broad definitions which are 
those of the Association of College and Research Libraries (ACRL) of the American Library 
Association and which are already in use in a number of Canadian colleges and universities 
(Van House et al. 1990). 

Reference Transaction - An information contact that involves the knowledge, use, 
recommendation, interpretation, or instruction in the use of one or more information sources 
by a member of the library staff. 

Directional or facilitative transaction - An information contact that facilitates the use of the 
library but does not involve the knowledge, use, recommendation, interpretation, or instruction 
in the use of any information sources. 

Reference transactions may be further categorized as "Basic" or "Complex". Basic level 
transactions include requests for factual information or requests for substantive information on 
a single subject which can be easily and quickly provided, generally with reference to one or 
two standard information/reference sources and including basic instruction if required. 

Complex transactions include requests for factual information or requests for substantive 
information on a complex subject which may require consultation of a variety of 
information/reference sources and/or the exercise of high-level information-seeking and 
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instructional skills. 



Clearly, complexity is difficult to define, although most reference librarians have an inherent, if 
unexpressed understanding of what it means. Your comments and suggestions on the issue 
of definition of transaction complexity will be especially welcome. 

The forms we have designed are variations on a theme. They measure two variables only, 
type of question and period of collection, i.e. daily (by hour), daily (by time of day), weekly, 
and monthly. Please select the format that best meets your needs. We have not attempted 
to incorporate in a single form the multitude of other variables that survey respondents 
currently measure. Again, your comments and suggestions will allow us to modify the base 
form. On the basis of pilot project responses and further analysis of survey results we will be 
making recommendations regarding variable selection and sampling techniques. 

Thank you again, for your willingness to take part in this project. Evaluation guidelines will be 
sent to you before the end of March and we look forward to your responses. If you have any 
questions about the information contained in this package please let me know. 



Sincerely, 



E. Jane Philipps, Chair 

CACUL Working Group on Reference Statistics 
Phone: (613) 545-6846 
Fax: (613) 545-6819 
E-mail: philippj@qucdn.ca 



Van House, Nancy A., Beth T. Weil, and Charles R. McClure. Measuring Academic Library 
Performances Practical Approach , Chicago: American Library Association, 1990. 
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CACUL Working Group on Reference Statistics 



V . 



Pilot Project 

March 1, 1993 to March 31, 1993 



Process 



1 . Review the definitions with all staff members who will be participating in the project; this 
should include all those who regularly provide information/reference service. 



2. Select the version of the form which best meets your need and make the number of 
double-sided copies required for the month of March. 

3. Place the form in a visible/accessible position at those desks where information/reference 
statistics are recorded. 

4. Using a tally |||| to record all transactions as facilitative, or as basic or complex reference 
during all those hours that information/reference service is regularly provided. 

5. If practicable, maintain a brief note of questions which are difficult to categorize on the 
basis of the definitions provided and of those features of the forms and definitions which meet 
your needs and of those features which are in any way problematic. 

6. On the basis of the month's experience complete the evaluation sheet which will be mailed 
to you as soon as possible. 
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APPENDIX 4 . Pilot project working definitions 



REFERENCE 

An information contact that involves the knowledge, use, recommendation, interpretation, or 
instruction in the use of one or more information sources. Information sources include printed 
and nonprinted materials, machine-readable databases (including computer-assisted 
instruction, catalogues, and other holdings records), through communication or referral, other 
libraries and institutions, and persons both inside and outside the library. 

BASIC: A request for factual information or a request for substantive information on a single 
subject which can be easily and quickly provided, generally with reference to one or two 
standard information/reference sources and including basic instruction if required. 

COMPLEX: A request for factual information or a request for substantive information on a 
complex subject which may require consultation of a variety of information/reference resources 
and/or the exercise of high-level information-seeking and instructional skills. 

FACILITATIVE 

An information contact that facilitates the use of the library but which does not involve the 
knowledge, use, recommendation, interpretation, or instruction in the use of any information 
sources. Facilitative transactions include "giving directions for locating staff, users, physical 
features within the library, or providing assistance (of a nonbibliographic nature) with 
machines" (Van House et al. 1990:96), and explaining library policies and procedures. 
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APPENDIX 6 . Pilot Project Evaluation Questionnaire 

CACUL Working Group on Reference Statistics 
Pilot Project - Evaluation 

Please provide detailed comments and suggestions wherever possible. 

A. DEFINITIONS 

1. The definitions clearly distinguish between reference and non-reference (directional or facilitative) 
transactions. Please identify any queries which were difficult to classify. 

Yes No 



Comments: 



2. The definitions clearly distinguish between basic and complex reference transactions. Please identify 
any queries which were difficult to classify. 



Yes No 



Comments: 



3. The definitions are appropriate and we would use them. Please suggest changes that would make 
the definitions more useful for you. 



As is With changes No 



Comments: 
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B. FORMS 



4. We would use the following form(s). Please attach examples of modified forms. 

As is With changes No 

a. Daily (by time of day) 

b. Daily (by hour) — 

c. Weekly — 

d. Monthly — 

Comments: 



5. We would prefer to collect statistics 






a. On a daily basis 


Yes 


No 


b. Using a valid sampling technique 


Yes 


No 


Comments: 







6. General Comments 



PLEASE RETURN BY APRIL 30 1993 
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APPENDIX 7 . Final report of the working group on reference statistics 



♦ 



» 



The CACUL Working Group on Reference Statistics was established by the CACUL Executive 
in January 1990 with the following mandate: 

• Develop definitions, forms and procedures, and conduct a pilot study, to estabush 

A BODY OF NORMATIVE DATA ON REFERENCE ACTIVITIES IN ACADEMIC LIBRARIES. 
THE MEMBERS OF THE WORKING GROUP ARE SANDRA BLACK (MOHAWK COLLEGE), ANDREA GLOVER 

(University of Lethbridge), Jim Kelly (University of Lethbridge), and Jane Philipps (Queen's 
University). The activities and recommendations of the Working Group are outlined below. 



DATA COLLECTION: 

The first task of the Working Group was to collect information on current activities relating to the 
collection and use of reference statistics in Canadian academic libraries. To that end 

• a questionnaire was developed and pretested by 12 college and 10 university 
libraries in December 1991. 

• the questionnaire, revised on the basis of pretest responses, was mailed as the 
"CACUL Survey of Activities Relating to the Collection, Reporting, and Use of 
Reference Statistics in Canadian Academic Libraries" to 76 college and 42 
university libraries in April 1992. 

The response rate of 78.6 percent from university libraries and 56.6 percent from college libraries 
provides a substantial body of data on reference activities. 



DEFINITIONS AND FORMS 

On the basis of the survey data, a literature review, and an examination of existing standards, 
the Working Group 

• developed definitions for 'Basic' and 'Complex' reference transactions and for 
'Facilitative' non-reference transactions 

• developed a set of forms to be used in recording information on reference activity 

• investigated sampling techniques 
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PILOT PROJECT 

To test the efficacy of definitions and forms 

• 14 college and 9 university libraries used the draft forms and definitions at their 
reference desks in March 1993 and 

• completed an evaluation of the forms and definitions. 

The definitions and forms were modified on the basis of responses from the pilot project 
participants. 



PRESENTATION OF RESULTS 

Members of the Working Group presented the results of the survey and the pilot project at the 
Working Group's own session at the Canadian Library Association Annual Conference in Hamilton 
in June 1993. The session was very well attended with wide representation from all types of 
libraries. The definitions were favourably received and subsequently published in Feliciter 
(November/ December 1993). 



RECOMMENDATIONS 

The CACUL Working Group on Reference Statistics recommends that 

• the CACUL Executive endorse the forms and definitions developed by the Working 
Group (attached), encourage by all possible means their adoption in all college 
and university libraries, and recommend to CLA that they be adopted as a national 
standard. 

• the results of the survey and pilot project be submitted to the CACUL Executive 
for publication as a CACUL Occasional Paper by June 1994; 

• the CACUL Working Group on Reference Statistics ceases to be a standing 
committee of CACUL as it has completed its mandate. 



SIGNED BY: 

Sandra Black Jim Kelly 

Andrea Glover Jane Philipps 
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